Construct and Criterion-related validation of the Bifactor Engagement Scale


John Kulas Mike DeFabiis Casey Osorio Morgan Russell



The roots of employee (sometimes aka work, e.g., Schaufeli & Bakker, 2010a) engagement research likely started with theoretical expansions of forms of employee participation (see, for example, Ferris & Hellier, 1984) and job involvement (e.g., Elloy et al., 1991). This exploration extended into broader considerations of attitudes and emotions (Staw et al., 1994) and were informed by further exploration of the dimensionality of constructs such as organizational commitment (Meyer & Allen, 1991). The 1990’s saw focused development and refinement. Staw et al. (1994) investigated the relationships between positive emotions and favorable work outcomes, and although they do not use the word, “engagement”, their distinction between felt and expressed emotion likely held influence upon the burgeoning interest in the engagement construct.

Our Measure of Engagement

Our theoretical conceptualization of work engagement is primarily informed by Schaufeli & Bakker (2003) and Rosenberg (1960). Through the lens of our framework, engagement is a mental state wherein employees: a) feel energized (Vigor), b) are enthusiastic about the content of their work and the things they do (Dedication), and c) are so immersed in their work activities that time seems compressed (Absorption). We further decompose each of these facets into three attitudinal components: d) feeling (e.g., affect), e) thought (e.g., cognition), and f) action (e.g., behavior).

Participants

Of the 743 total Qualtrics panel respondents, roughly half were excluded based on conservative indices of carelessness across the larger survey. These screens included respondents with more than 50% missing responses, those who provided consistently non-differentiating responses across more than 12 consecutive items, and those who completed the survey in less than 300 seconds. These conservative screens resulted in a retained validation sample of 377. All analyses were derived from this n of 377.

Data analysis

We used R (Version 4.2.0; R Core Team, 2022) and the R-packages careless (Version 1.2.1; Yentes & Wilhelm, 2021), corrplot2021 (Wei & Simko, 2021), papaja (Version 0.1.0.9999; Aust & Barth, 2022), psych (Version 2.1.9; Revelle, 2022), and tinylabels (Version 0.2.3; Barth, 2022) for all our analyses. As a straightforward validation study, our analyses consisted predominantly of Pearsons product-moment correlations.

Results

Table 1:
Suggested final scale definitions.
Substantive Attitudinal Item.Number Item.Stem
Absorption Cognitive 1 I am able to concentrate on my work without getting distracted
Absorption Cognitive 3 Time passes quickly while I’m working
Absorption Affective 5 I enjoy thinking about work even when I’m not at work
Absorption Affective 8 I love starting my workday
Absorption Behavioral 10 I have to be reminded to take breaks while I’m at work
Absorption Behavioral 11 I never miss a work deadline
Vigor Cognitive 14 Thinking about work saps my energy
Vigor Cognitive 16 I’m able to maintain good levels of energy throughout the workday
Vigor Affective 17 I enjoy spending time completing my job tasks
Vigor Affective 19 I feek motivated to go beyond what is asked of me at work
Vigor Behavioral 21 When work is slow I find ways to be productive
Vigor Behavioral 22 I express enthusiasm for my job while at work
Dedication Cognitive 25 I plan to stay with this company as my career advances
Dedication Cognitive 26 I believe this company cares about my career goals
Dedication Affective 31 I feel proud of my accomplishments within this organization
Dedication Affective 32 My job makes me feel like I’m part of something meaningful
Dedication Behavioral 34 I embrace challenging situations at work
Dedication Behavioral 35 I speak positively about this organization to others

Note. The recommended response scale is ‘Strongly Disagree’, ‘Disagree’, ‘Somewhat Disagree’, ‘Somewhat Agree’, ‘Agree’, and ‘Strongly Agree’. TAKE THE ITEM # COLUMN OUT BEFORE FINAL POSTER

 

The items comprising the focal measure along with their scale associations and recommended administered response scale are located in Table 1. The current sample internal consistency estimates for our three substantive subscales were: 1) Absorption (\(\alpha\) = 0.75), 2) Dedication (\(\alpha\) = 0.89), and 3) Vigor (\(\alpha\) = 0.75), and estimates for our three attitudinal subscales were: 1) Affect/“Feel” (\(\alpha\) = 0.86), 2) Behavior/“Do” (\(\alpha\) = 0.77), and 3) Cognition/“Think” (\(\alpha\) = 0.77).

Construct validation

For convergent validity indices, we administered the 17-item Utrecht Work Engagement Scale (Schaufeli et al., 2002; Schaufeli & Bakker, 2010b) as well as Saks (2006)’s 12-item measure which aggregates to two scales: job and organizational engagement (see also Saks, 2019).1 An example item from the Saks (2006) (job) scale is, “Sometimes I am so into my job that I lose track of time”. An example item from the Schaufeli et al. (2002) scale is, “At my work, I feel bursting with energy”. The Schaufeli et al. (2002) measure follows the same structure as our focal measure, so we aggregated to subscales of Absorption (\(\alpha\) = 0.84), Dedication (\(\alpha\) = 0.87), and Vigor (\(\alpha\) = 0.85). Internal consistency estimates for the Saks scale were \(\alpha\) = 0.69 (job engagement) and \(\alpha\) = 0.84 (organizational engagement). Also note here that the English version of the UWES may actually be a translation (it is difficult to say for sure, as the test manual describes an original Dutch sample although the manual is written in English). Further suggesting that the English version may be a translation, some items have odd grammar (for example, “I am proud on [sic] the work that I do”).

Two short scales from the Oregon Avocational Interest Scales (Goldberg, 2010) were retained for discriminant validitation - the 5-item “Pets” and 5-item “Household Activities” scales. These items asked how frequently respondents engaged in different activities. An example Household Activity item is, “Cleaned the house” (current sample \(\alpha\) = 0.72) and an example Pets item is “Fed a pet animal” (current sample \(\alpha\) = 0.88).

References

  1. We had also intended to use the Gallup “Q12” for construct validation (Harter et al., 2013; Thackray, 2005), but Gallup was not willing to share item- or person-level data.↩︎